The process of sorting data is very important in data processing and, in particular, it is important in the processing of commercial and scientific processes. In both types of processes, clock time can be a critical factor in providing end results. In both types of processing, use and availability of computing resources is a critical factor, which directly affects meeting deadlines. In both types of processes, there has been an ever increasing need of accuracy in end results and an ever increasing volume of data to support the production of end results. Consequently, sorting of data is taking considerable time and resources in the commercial and scientific computing environments.
There are several ways to approach the problem of reducing time and resources for sorting data. These approaches include improving sorting techniques, integrating sorting functions in hardware, enabling parallel sorting, and providing better use of computing resources (e.g., in-memory sorts, faster I/O paths, and processors dedicated to sorting).
Another technique for reducing the time and resources used in sorting includes using a sort process as a stage in a pipeline with the input to sort being read from a pipe and the output from sort being written to the pipe. This method reduces I/O latencies inherent in the use of peripheral storage and enables some reduction in elapsed time by providing for parallelism with the process that writes the data to sort and sort input phase processing, and between the sort output phase and the process that reads the sorted data.
However, a need still exists for a mechanism to enhance parallelism for the entire sorting process, thereby, reducing the elapsed time. In particular, a need exists for enabling the writer to sort (e.g., Process A), the sort (SORT), and the reader of the sorted data (e.g., Process B) to execute in parallel, thus, improving the elapsed time of the entire process. A need also exists for further reducing the resources necessary to perform a sort, and to reduce the number of records sorted at one particular time.